Dynamic bipedal walking assisted by learning

This thesis proposes a general control architecture for 3D dynamic walking. It is based on a divide-and-conquer approach that is assisted by learning. No dynamic models are required for the implementation. In the approach, the walking task is divided into three subtasks: 1) to maintain body height; 2) to maintain body posture; and 3) to maintain gait stability. The first two subtasks can be achieved by simple control strategies applied to the stance leg. The third subtask is mainly effected by swing leg behavior. By partitioning the overall motion into three orthogonal planes (sagittal, frontal and transverse), the third subtask can be further divided into forward velocity control (in the sagittal plane) and lateral balance (in the frontal plane). Since there is no explicit solution for these subtasks, reinforcement learning algorithm, in particular, Q-learning with CMAC as function approximator, is used to learn the key parameters of swing leg strategy which determine the gait stability. Based on the proposed architecture, different control algorithms are constructed and verified using the simulation models of two physical bipedal robots. The simulation analysis demonstrates that by applying an appropriate local control laws at stance ankle joints, the learning rates for the learning algorithms can be tremendously improved. For the lateral balance, a simple control algorithm (without learning) based on the notion of symmetry is discovered. The stability of the control algorithm is achieved by applying appropriate local control law at the stance ankle joint. A dynamic model is constructed to validate this approach. The control algorithms are also demonstrated to be general in that they are applicable to different bipeds that have different inertia and length parameters. The designer is also free to choose the walking posture and swing leg strategy. Both "bird-like" and "human-like" walking postures have been successfully implemented for the simulated bipeds. Thesis Supervisor: Gill A. Pratt Title: Professor of Electrical Engineering and Computer Science